Choosing a new cosmetic product is often a tedious and time consuming process, and is only usually possible in a retail environment where samples are made available. An important consideration for a customer trying on any new product is seeing how it looks as they move around, taking momentary opportunity to view themselves wearing the cosmetic from particular angles or with particular expressions.
Utilising the mass availability of handheld, or other, computing devices to make real-time virtual try-on of new cosmetics possible in any environment has the potential to radically change the way the customer finds the perfect product. Three main challenges for any such system are first, locating and tracking the features of a subject in a live captured image data stream, second, augmenting a virtual cosmetic product realistically in place over the live images, and finally to do all this in real-time, particularly on devices having limited hardware capabilities.
Feature tracking systems are generally known, in which tracking of an identified person or object in a captured scene is performed based on established image processing techniques. For example, one well known technique for object shape modelling is Active Shape Modelling (ASM), as discussed for example in “Lip Reading Using Shape, Shading And Scale”, Mathews, Cootes, Cox, Harvey and Bangham, and “An Introduction To Active Shape Models”, Tim Cootes. Another well known technique for object appearance modeling is Active Appearance Modelling (AAM), as discussed for example, in “Active Appearance Models”, Cootes, Edwards and Taylor.
However, conventional feature tracking systems are not efficient and typically require significant computational overheads, for example for off-line training of object models and subsequent live tracking of objects based on the trained models. Moreover, techniques such as AAM perform well on conventional per-person models but are slow, not robust enough and unable to generalize for data not included in the training set.
What is desired are improved techniques for feature tracking and augmenting that address these challenges.